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Achievement Effects of Embedded Multimedia 
In a Success for All Reading Program 

Abstract 

Embedded multimedia refers to teaching methods that embed video content 
within teachers’ lessons. The research of Mayer and others has suggested that 
multimedia instruction can enhance learning by using the capacity of both visual and 
verbal memory systems. The present study is an evaluation of embedded multimedia in a 
year-long randomized clinical trial comparing first graders who learned beginning 
reading using the Success for All program either with or without embedded, brief video 
components. A study involving 394 first graders in 10 high-poverty schools found 
significant positive effects on the Woodcock Word Attack scale, controlling for pretests, 
in HLM analyses with school as the unit of analysis. The results provide partial support 
for the utility of embedded multimedia as a component of beginning reading instruction. 
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For more than 50 years, educators and policymakers have been expectantly 
waiting for the video* revolution in education. Indeed, research on educational programs 
such as Sesame Street (Bogatz & Ball, 1971; Fisch & Truglio, 2000; Rice, Huston, 
Truglio, & Wright, 1990) and Between the Lions (Linebarger, Kosanic, Greenwood, & 
Doku, 2004) has shown positive effects of educational television for the reading and 
language development of young children. Yet video has remained a minor medium in the 
classroom, where it has been seen as a replacement for teacher instruction rather than a 
tool for teachers. 

In recent years, however, video has begun to appear in educational practice in a 
new form that has great potential for education reform. Video is one example of what 
Mayer (2001) calls “multimedia,” instructional formats that combine words and pictures. 
Instead of replacing instruction, multimedia can be embedded in classroom instruction to 
enhance teachers’ lessons. This “embedded multimedia” application is not widely known 
or used, but research on the practice has shown initial promise. In embedded multimedia, 
brief multimedia segments are threaded throughout teachers’ lessons. In one sense, 
embedded multimedia should at least share the impacts on achievement found in studies 
of the use of pictures, illustrations, diagrams, and other graphic content to enhance the 
effects of class lessons and text (see Schnotz, 2002; Shah, Mayer, & Hegarty, 1999; 
Carney & Levin, 2002; Vekiri, 2002). Yet well-designed embedded multimedia using 
video has properties that go beyond static graphics. First, video can model skills or 
content for students, giving them clear demonstrations of proficient performance. For 
example, videos can show children modeling advanced problem-solving strategies, 

* Throughout this paper, the term “video” is assumed to include DVD and CD formats as well as VHS 
unless otherwise noted. 
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sounding out words, thinking out loud about their creative writing, or working through a 
scientific investigation. Further, any multimedia that models for children also models for 
teachers, providing “just-in-time” professional development. For example, showing 
children working in cooperative groups to solve particular kinds of problems may provide 
teachers with a clear idea of what cooperative learning should look like, just as it would 
for students. 

Theoretical Background 

The theoretical basis for embedded multimedia begins with Paivio’s dual coding 
theory (Clark & Paivio, 1991), which demonstrated that information held both in verbal 
memory and in visual memory is retained better than information held in only one 
memory system. For example, Mayer (2001) and his colleagues carried out a series of 
experiments in which students were taught how lightning works. Teaching about 
lightning using narration alone or text alone teaches only to verbal memory. Adding 
diagrams or moving pictures to illustrate the concept teaches to visual memory. Adding 
the pictures to narration or text greatly increased initial learning of the concept, as well as 
both retention and transfer, as long as the pictures and text or narration were closely 
aligned with each other and focused on the instructional objective (Mayer & Moreno, 
2003). 

Another key concept in multimedia learning is cognitive load. It has long been 
known that working memory places a severe limitation on the amount of information that 
can be absorbed at any given time (Solso, 2001). Each learning objective has a given 
intrinsic cognitive load, and if this exceeds the working memory capacity of children of a 
given age or level of knowledge, the objective must be broken into components that “fit” 
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within the child’s working memory capacity (Paas, Renkl, & Sweller, 2003). However, 
each memory system has its own limitations, which may be fairly independent of one 
another. That is, a learner has a limited working memory capacity for words and a limit 
for pictures, but “using up” the word limit does not “use up” the picture limit. Therefore, 
Mayer and Moreno (2003) suggest that instructors or designers can “off-load” 
meaningful information from one channel to the other, by using fewer words and more 
pictures when verbal working memory would otherwise be overloaded. Again, this “off- 
loading” only works if the pictures and words directly support each other. Pictures that 
are interesting but irrelevant to the words, containing “seductive details,” can be 
detrimental because they fill limited working memory with the irrelevant content 
(Mayer, 2001). 

The complementary nature of words and moving pictures has been demonstrated 
in a series of studies in which instructional video material was broken into audio and 
silent video components and compared to the full video with audio (see Kozma, 1991, for 
a review). In most cases children learned significantly more from the video-audio 
presentations, and in no case did they learn less. This supports the concept that narrative 
and picture information do not interfere with each other, but instead support each other. 

When both memory systems are in danger of overload, Mayer and Moreno’s 
(2003) research suggests segmentation of combined word-picture content, organizing 
content in “bite-size segments” with opportunities to integrate and organize new 
information between segments (Mayer & Chandler, 2001). 

Embedded multimedia takes advantage of these theoretical principles, combining 
verbal and visual content (words and moving pictures) to give learners multiple pathways 
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to retention and comprehension and interspersing “bite-sized segments” with 
opportunities to practice and apply new learnings. For example, a key problem in early 
reading instruction is teaching associations between letter shapes and letter sounds. The 
embedded multimedia strategy described in this paper contains a series of animations 
designed to link the letter shape and letter sound in a brief, memorable story. In 
introducing the /p/ sound, for instance, a parrot eats some watermelon, and then spits out 
the seeds, making the sound /p/, /p/, /p/. The seeds fall in the shade of the letter “p”. 

This 30-second story, which five-year-olds readily leam and remember, gives them a link 
in their visual memory between the shape “p” and the sound /p/ that they can easily 
access, adding to the verbal and tactile learning the teacher provides by having children 
look at a picture of the parrot in the shape of the letter “p” and having them make the /p/ 
sound and noticing how their lips form when they do so. 

In English reading, children must learn 44 phonemes represented by more than 60 
graphemes (Adams, 1990). In order to quickly and automatically recognize each, and 
avoid confusions among them, children need to practice them separately and in words 
and sentences. Practice in cooperative groups, working with the teacher, and practice in 
other formats, is necessary to solidify learnings from “bite-sized segments,” which is why 
the multimedia segments by themselves are insufficient. 

Similar theories underlie other multimedia applications in beginning reading (see 
Pailliotet & Mosenthal, 2000). To teach sound blending, the present experiment used 
video skits showing puppets sounding out words, creating both visual and auditory 
representations in memory. To teach vocabulary, live-action skits acted out the meanings 
of key words, both in context and then out of context. Again, students were expected to 
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recall the skits (in their visual memories) to gain access to word meanings (needed in 



their verbal memories). 



Success for All and Embedded Multimedia 

In 2001, researchers and developers at the nonprofit Success for All Foundation 
and Johns Hopkins University began a project to enhance the Success for All reading 
program with embedded multimedia. Success for All teaches beginning reading using a 
systematic phonics approach (see Slavin & Madden, 2001, for a program description). 
Fifty experimental-control comparisons of one or more years’ duration have found, on 
average, positive effects of Success for All on children’s reading achievement (see 
Borman, Hewes, Overman, & Brown, 2003; Herman, 1999; Slavin & Madden, 2001). 

The addition of embedded multimedia to Success for All’s beginning reading 
program was intended to enhance the effectiveness of the program by giving children 
compelling, memorable demonstrations of letter sounds, sound blending strategies, 
vocabulary, and comprehension strategies. A particular focus was on the needs of 
English language learners, who were felt to be in particular need of visual models for 
vocabulary and sound blending. 

A preliminary study of the embedded multimedia strategy was carried out by 
Chambers, Slavin, Madden, Cheung, and Gifford (2004). In that study, four schools 
primarily serving Hispanic students were compared with four matched schools with 
similar demographics and achievement histories. The experimental schools used Success 
for All with embedded multimedia, while the control schools used traditional basal 
approaches. After a one-year implementation, students in the Success for All/Embedded 
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Multimedia treatment scored significantly higher than those in the matched control group, 
controlling for Peabody Picture Vocabulary Test pretests, on the Woodcock Word 
Identification, Word Attack, and Passage Comprehension scales. There were no 
differences on Letter Identification. 

The Chambers et al. (2004) experiment established the combined effect of 
Success for All reading and embedded multimedia on reading achievement, but it did not 
allow for a test of the unique contribution of the embedded multimedia content. 

The Present Study 

The present study builds on the findings of Chambers et al. (2004) study in 
several ways. First, it controlled the instructional content. All schools implemented 
Success for All reading, so that experimental and control schools differed only in the use 
or non-use of the multimedia content. All schools were in the same district, the Hartford 
(CT) Public Schools, further ensuring similarity between experimental and control 
schools on all factors other than the embedded multimedia treatment. In addition, in the 
present study schools were randomly assigned to the embedded multimedia or control 
treatments. Random assignment eliminated selection bias from the experiment (see 
Mosteller & Boruch, 2003), as all schools had an equal chance of being assigned to the 
experimental or control groups. All school staffs were willing to implement the 
embedded multimedia program; those randomly assigned to the control group were given 
the materials and training in the year following the experiment, as a delayed treatment 
control procedure. 
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Methods 



Design 

This study employed a cluster randomized trial (CRT) design, with random 
assignment of schools to treatments. A total of 10 elementary schools were identified. 
Five were randomly assigned to standard Success for All (control) and five to Success for 
All with embedded multimedia. 

Participants 

Subjects were 450 first graders, of whom 394 completed pre- and posttests. The 
students attended 10 schools in inner-city Hartford, CT. Almost all students qualified for 
free- or reduced-price lunches, and 62% of students were Hispanic and 35% African- 
American. As indicated in Table 1, the two groups were well matched on the percentage 
of students eligible for free or reduced lunch, ethnicity, and the percentage of limited 
English proficient students in the school. Pretests were used as covariates in the main 
analyses to adjust for any initial difference between the two groups and to increase 
statistical power. Because the unit of random assignment and treatment was the school, 
the school was also the unit of analysis. 



Insert Table 1 here 
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Treatments 



Success for All with Embedded Multimedia. The experimental group used 
multimedia content embedded in teachers’ daily 90-minute Success for All reading 
lessons. This material consisted of 30-second to three-minute skits and other 
demonstrations integrated with teachers’ lessons. No additional time was added to the 
lessons to accommodate the multimedia. The purpose of the multimedia content was to 
directly present to students compelling demonstrations of key elements of beginning 
reading: letter sounds, sound blending strategies, and vocabulary. In addition, it was 
hoped that by showing multimedia content in class, teachers would have constant 
reinforcement of effective teaching strategies. 

The multimedia materials ultimately created for the program were called Reading 
Reels. Reading Reels includes the following components. 

The Animated Alphabet . Animations teach and reinforce sound/symbol 
relationships. For example, the animation introducing the short /e/ sound features an 
elephant pushing a rock with an “e” on it up a hill, making an /e/ sound with each 
push. At the top, the rock rolls down, and the exhausted elephant says “ehhhh” in 
frustration. The pairing of the memorable images, the letter sound, and the letter 
shape gives students many mental pathways to link the letter with its sound. There 
are animations for 58 different graphemes that comprise most of the phonemes used 
in the English language. Each animation is between 30 seconds and one minute long. 

The Sound and the Furry . Multimedia skits, using SFA puppet characters, model 
the word blending process, phonemic awareness, spelling, fluency, reading strategies, 
and cooperative learning routines. For example, a puppet sees a sign, “Watch out for 
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stick.” He sounds out the word “stick” phonetically. Then he notices a stick, which 
he picks up. The stick sticks to his fur, and in trying to get it off he bites it — and then 
realizes he’s in real trouble. After the skit, the sounding out section is repeated, and 
children sound out the word along with the puppets. More than a hundred such 
vignettes illustrate sound blending strategies from simple CVC words to multi- 
syllable words. The average puppet skit is about two minutes long. 

Word Plays . Live action multimedia skits dramatize important vocabulary 
concepts from the Success for All beginning reading texts. These skits are 
particularly designed to help English language learners build the specific vocabulary 
for the books they will be reading. For example, when children are about to read a 
story about China, they first see a skit that introduces words such as “chopsticks,” 
“fireworks,” “beautiful,” and “ugly.” The average Word Play is about three minutes 
long. 

Between the Lions . Puppet skits and animations from the award-winning PBS 
program help teach phonemic awareness, sound/symbol correspondence, and sound 
blending. Between the Lions segments are about one minute long. 

The embedded multimedia materials were developed to be particularly beneficial 
to English language learners. In particular, the vocabulary skits demonstrating the 
vocabulary emphasized in each story children are about to read were designed to 
ensure that all children know the vocabulary in advance so that they can focus on the 
decoding and comprehension tasks required to master and enjoy each book. The 
alphabet and sound-blending segments are also designed to build children’s language 
skills as well as their reading skills. 
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Control. Control schools used the regular 90-minute Success for All reading 
program without the embedded multimedia content. In place of the multimedia 
content, teachers used picture cards to illustrate the letter shapes and vocabulary in 
the student books. Demonstrations and games were used to teach word-level 
blending. 

Measures 

Participants were pretested in early October, 2003, and posttested in early May, 
2004. The pretests were the Peabody Picture Vocabulary Test (PPVT) and the Word 
Identification subtest from the Woodcock Reading Mastery Test-Revised (WMRT-R). 
On average, each testing session took approximately 30 minutes per child. 

The posttests included the reading fluency test from the Dynamic Indicators of 
Basic Early Literacy Skills (DIB ELS) (Good & Kaminski, 2002) and three scales from 
the WRMT-R: Word Identification, Word Attack, and Passage Comprehension. Testing 
sessions required, on average, 42 minute per child. 

The DIB ELS fluency test is a standardized, individually administered test of 
accuracy and fluency with connected text. Student performance is measured by having 
students read a passage aloud for one minute. The Woodcock Word Identification test 
requires the subject to identify isolated words that appear in large type on the subject 
pages in the test book. The Word Attack test requires the subject to read nonsense words 
(letter combinations that are not actual words). The test measures the subject’s ability to 
apply phonic and structural analysis skills in order to pronounce words with which he or 
she is unfamiliar. The Passage Comprehension test measures the subject’s ability to read 
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a short passage — usually two to three sentences long — and to identify a key word missing 
from the passage. 

Analyses 

The data were analyzed using Hierarchical Linear Modeling (HLM; Raudenbush, 
1997), with students nested within schools. Classroom-level analyses could not be done 
because, as part of the Success for All design, children routinely change reading teachers 
over the course of the year. HLM is the appropriate, conservative analysis for cluster 
randomized designs of this kind. Condition was the independent variable and the 
WRMT-R subtests and the DIBELS fluency test were the dependent measures. The 
PPVT and Word ID pretests were used as covariates to adjust for initial differences 
between the treatment and control groups and to increase statistical power. Analyses 
were carried out both for the overall sample and for a Hispanic subsample. 

Results 

Pretests 

Pretests, summarized in Table 2, showed that experimental and control schools 
were well-matched on the PPVT, but the control schools scored higher at pretest on the 
Word Identification pretest than the experimental schools, with an effect size of -0.35. 
This was statistically significant at the individual level (p< .01), but was not significant at 
the school level. 
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Insert Table 2 here 



Hierarchical Linear Modeling 

For the hierarchical linear analyses, we developed 2-level hierarchical models that 
nested students and their pretest and posttest scores within schools. The fully specified 
level 1 , or within-school model, nests students within schools with pretests as covariates. 
The linear model for level 1 model is written as 



Yij=p 0 j + Pij (pretest) + rij 



Where 

Yij is the expected subtest score of child i in school j (j=l 10 schools); 

Poj is the mean outcomes in school j after controlling for pretest difference; 
Py is the random level - 1 pretest effect. 

The fully specified level 2 model is written as 

Poj=8oo + Soi (Treatment) + 802 (Mean Pretest) + Uqj 

Pij=Sio 



where 

§oo is the mean achievement in the control schools; 

§oi is the treatment effect; 

802 is the school-level mean pretest effect; 

810 is the pooled within-school regression coefficient for the level -1 covariate. 

Because the pretest slope was no different across all schools on all four measures, 
it was treated as fixed at level 2 of the model and no attempt was made to model the 
effect with school-level predictors. The intraclass correlation was +.04. 



13 




As indicated in Tables 3 and 4, the experimental group scored significantly higher 
than the control group on the Word Attack subtest. Using the treatment coefficient of 
6.21 for the mean achievement outcome and dividing this value by the unadjusted 
standard deviation for the Word Attack posttest, we estimated the magnitude of this 
school-level effect at ES= +0.47. Although the experimental group also scored higher 
than the control on Word ID (ES=+0.23), Passage Comparison (ES=+0.20), and DIBELS 
(ES=+0.29), these differences were not statistically significant. Analyses for the 
Hispanic subsample produced results very similar to those of the whole sample. No 
interactions between ethnicity and treatment were found. 



Insert Tables 3 and 4 here 



Discussion 

The results of the current study partially support the expectation that the addition 
of embedded multimedia content to a beginning reading program would enhance 
children’s reading achievement. Schools were randomly assigned to conditions and all 
schools used the same Success for All reading strategies, so the use of embedded 
multimedia content was the only factor differentiating experimental and control 
conditions. Using a conservative HLM analysis, only one of the four outcome measures, 
Word Attack, showed significant experimental-control differences, but this is in line with 
theoretical expectations. Three of the four multimedia segments dealt primarily with 
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letter sounds and sound blending, which are key components of Word Attack. The 
fourth. Word Plays, focused on vocabulary. The other measures, especially Passage 
Comprehension and DIBELS, are more logically related to reading of connected text, 
which was emphasized equally in both treatments. 

Because of the use of Word Plays and the visual content in all of the Reading 
Reels materials, it was expected that Hispanic children would benefit disproportionately 
from the embedded multimedia treatment. However, this was not the case. Effects for 
Hispanic children were nearly identical to those for other children (mostly African 
American). However, no measures of English vocabulary were used, and we were unable 
to obtain data on children’s initial English proficiency, so it may be that there were 
undetected differential positive effects for the subgroup of children with limited English 
proficiency. 

The findings of the present study and of Chambers et al. (2004) suggest that the 
use of embedded multimedia has potential to enhance the effectiveness of beginning 
reading instruction for disadvantaged children. The individual-level effect size of +0.32 
for Word Attack (school level ES = +0.47) is educationally important, particularly given 
the minimal cost of the intervention. Adding the DVDs to an existing Success for All 
school costs $129 per classroom (plus the cost of a DVD player and television) and 
occupies an average of less than five minutes per day out of a 90-minute reading period. 
However, it is important to note that across all four measures, the individual-level mean 
effect size was +0.17 (mean school-level ES = +0.30). 

Much research remains to be done to further understand these effects. First, a 
larger experiment, involving at least 40 schools, is needed to provide sufficient statistical 
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power for the HLM analysis (Raudenbush, 1997). Such studies should add measurement 



of vocabulary outcomes, especially for English language learners. Further, it would be 
important to study the program’s effects on teachers’ practices, to assess the validity of 
the theory that multimedia modeling of effective teaching practices would improve 
teachers’ implementations of the reading program and their overall teaching skill. 

Further, smaller-scale studies are needed to examine the separate impacts of the 
multimedia components, and to vary elements of the interventions. Such studies could 
help build a theoretical base for embedded multimedia in reading instruction and suggest 
design principles for further development. 

Research on multimedia learning (e.g., Mayer, 2001) shows that specific design 
elements matter a great deal in determining learning outcomes. However, using design 
principles known to contribute to learning, multimedia embedded in literacy instruction 
may have significant potential to improve reading outcomes for children. 
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Table 1 

Demographic Characteristics of Experimental and Control Schools 




Experimental 


Control 


Number of schools 


5 


5 


Number of students 


189 


205 


% free lunch 


99 


95 


Ethnicity: 






% Hispanic 


61 


62 


% African-American 


35 


35 


% White 


4 


2 


% LEP (schoolwide) 


29 


31 
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Table 2 

Individual-Level Pretests (Fall, 2004) 





Experimental (N=189) 


Control (N=205) 




Pretests 


Means 


SD 


Means 


SD 


Individual- 












Level ES 


PPVT 


86.78 


11.33 


88.07 


10.44 


-0.12 


Word ID 


381.74 


27.01 


390.14 


24.06 


-0.35** 



** pc. 01 (individual-level analysis; differences were not significant in school-level 
analyses.) 
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Table 3 

Individual-Level Posttests (Spring, 2004) 

Experimental (N= 189) Control (N=205) Individual- 

Level ES C 





Post 


SD a 


Adjusted 

Post b 


Post 


SD a 


Adjusted 

Post b 




Word ID 


425.3 


23.0 


428.9 


427.8 


22.5 


424.9 


+0.15 


Word 


480.5 


13.7 


481.7 


478.7 


12.7 


477.6 


+0.32 


Attack 
















Passage 


456.5 


15.3 


458.2 


458.6 


15.2 


457.0 


+0.08 


Comp. 
















DIB ELS 


29.0 


23.9 


32.1 


32.3 


24.5 


29.4 


+0.12 



a Standard deviation of unadjusted means 
b Adjusted for PPVT and Woodcock Word Identification 

c Effect Size=Adjusted Posttest (exp) - Adjusted Posttest (control)/Unadjusted SD 
(control) 
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Table 4 



Hierarchical Linear Modeling (HLM) Analyses 



Posttests 




Coefficient 


SE 


p-value 


School 
Level ES 


Word ID 


Intercept 800 


427.78 


1.35 


0.000 






Treatment 801 


5.11 


3.17 


0.151 


+0.23 




Pretest S 02 


0.907 


0.22 


0.005 




Word Attack 


Intercept 800 


480.32 


0.997 


0.000 






Treatment 8 0 i 


6.21 


2.32 


0.032 


+0.47* 




Pretest S 02 


0.49 


0.16 


0.020 




Passage Comp. 


Intercept 8 0 o 


458.35 


1.03 


0.000 






Treatment 801 


3.04 


2.4 


0.246 


+ 0.20 




Pretest S 0 2 


0.629 


0.16 


3.761 




DIB ELS 


Intercept 800 


31.96 


1.65 


0.000 






Treatment 8 0 i 


6.95 


3.85 


0.113 


+0.29 




Pretest 802 


1.21 


0.269 


0.003 





*p<0.05 
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